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ABSTRACT 



Apparatus and methods are disclosed for simultaneously 
tracking multiple finger and palm contacts as hands 
approach, touch, and slide across a proximity-sensing, 
compliant, and flexible multi-touch surface. The surface 
consists of compressible cushion, dielectric, electrode, and 
circuitry layers. A simple^rjrmimi^iransduction circuit is 
placed under each electrode to maximize signal-to-noise 
ratio and to reduce wiring complexity. Such distributed 
transduction circuitry is economical for large surfaces when 
implemented with thin-film transistor techniques. Scanning 
and signal offset removal on an electrode array produces 
low-noise proximity images. Segmentation processing of 
each proximity image constructs a group of electrodes 
corresponding to each distinguishable contact and extracts 
shape, position and surface proximity features for each 
group. Groups in successive images which correspond to the 
same hand contact are linked by a persistent path tracker 
which also detects individual contact touchdown and liftoff. 
Combinatorial optimization modules associate each con- 
tact's path with a particular fingertip, thumb, or palm of 
either hand on the basis of biomechanical constraints and 
contact features. Classification of intuitive hand configura- 
tions and motions enables unprecedented integration of 
typing, resting, pointing, scrolling, 3D manipulation, and 
handwriting into a versatile, ergonomic computer input 
device. 

118 Claims, 45 Drawing Sheets 
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To still further achieve the objects, the present invention 
comprises a 

multi -touch surface apparatus for sensing diverse 
configurations and activities 

of touch devices and generating integrated manual input to 
one of an electronic 

or electro-mechanical device, the apparatus comprising: an 
array of one of the 

proximity sensing devices described above; a dielectric cover 
having symbols 

printed thereon that represent action- to-be-taken when 
engaged by the touch 

devices; scanning means for forming digital proximity images 
from the array of 

sensing devices; calibrating means for removing background 
offsets from the 

proximity images; recognition means for interpreting the 
configurations and 

activities of the touch devices that make up the proximity 
images; processing 

means for generating input signals in response to particular 
touch device 

configurations and motions; and communication means for 
sending the input 

signals to the electronic or electro-mechanical device. 

To still even further achieve the objects, the present 
invention comprises a 

method for choosing what kinds of input signals will be 
generated and sent to 

an electronic or electro-mechanical device in response to 
tapping or sliding of 

fingers on a multi- touch surface, the method comprising the 
following steps: 

identifying each contact on the surface as either a thumb, 
fingertip or palm; 
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measuring the times when each hand part touches down and 
lifts off the surface; 

forming a set of those lingers which touch down from the all 
finger floating 

state before any one of the fingers lifts back off the 
surface; choosing the 

kinds of input signals to be generated by further distinctive 
motion of the 

fingers from the combination of finger identities in the set; 
genera tinng input 

signals of this kind when further distinctive motions of the 
fingers occur; 

forming a subset any two or more fingers which touch down 
synchronously after 

at least one finger has lifted back off the surface; choosing 
a new kinds of 

input signals to be generated by further distinctive motion 
of the fingers from 

the combination of finger identities in the subset; 
generating input signals of 

this new kind when further distinctive motions of the fingers 
occur; and 

continuing to form new subsets, choose and generate new kinds 
of input signals 

in response to liftoff and synchronous touchdowns until all 
fingers lift off 
the surface. 

processing means for generating input signals in response to 
particular touch 

device configurations and motions; and 

92. The method of claim 91, wherein production of discrete 
key symbols or 

mouse button click commands from single finger taps or finger 
chord taps is 

accompanied by transmission of activation signals to a light 
or sound feedback 
generating device. 

105. A method for choosing what kinds of input signals will 
be generated and 

sent to an electronic or electro-mechanical device in 
response to tapping or 

sliding of fingers on a multi- touch surface, the method 
comprising the 
following steps: 
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113. The method of claim 105, wherein generation of input 
signals is 

accompanied by generation of activation signals to a light or 
sound generating 

feedback device, and wherein the activation signals depend 
upon the kinds of 

input signals currently selected. 
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